Module 11: Cross-validation and the Control of Error Rates

This module emphasizes what might be termed “the practice of safe statistics.” The discussion is split into three parts: (1) the importance of cross-validation for any statistical method that relies on an optimization process based on a given data set (or sample); (2) the need to exert control on overall error rates when carrying out multiple testing, even when that testing is done only implicitly; (3) in the context of “big data” and associated methods for “data mining,” the necessity of some mechanism for ensuring the replicability of “found results.”